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ABSTRACT 



This study looked at the relationship between the programs 
students chose upon college entry, the programs from which they graduated, 
and the time taken to graduate. Individual student data on more than 204,000 
freshmen entering 38 public, land grant, and Research I universities in 1988 
and 1990 were collected. Descriptive statistics were used to summarize 
student entry characteristics such as college admission test scores and high 
school grade point averages. Logistic analysis was used to calculate and 
compare predicted and actual graduation rates based on student entry 
characteristics. Among findings were: graduation rates varied more by 
university than by program; time to completion varied by academic program; 
business and social sciences were the program areas which experienced the 
largest in-migration from other areas; students entering education programs 
had the lowest average Scholastic Assessment Test scores whereas engineering 
students had the highest; students initially undecided about their major were 
no less likely to graduate than other students and their graduation was not 
delayed; there was a correlation of about .28 between actual and predicted 
graduation rates; and there were significant differences among institutions 
between actual graduation rates and predicted graduation rates. Tables 
provide detailed findings for each institutions. (DB) 
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GRADUATION RATES: DO STUDENTS' ACADEMIC 
PROGRAM CHOICES MAKE A DIFFERENCE? 



Introduction 

In this paper, the results of the second phase of a multi-phase study which 
attempts to look at the graduation rates of first-time, full-time freshmen at public 
land-grant and research universities are discussed. At the Boston Forum in 
1995, the authors of this paper presented the results of the first phase of the 
study. In fact, two papers were presented that year about the study. In one, the 
methodology used to collect the data was outlined. In the second (Kroc, 
Woodard, Howard and Hull, 1995), the results of Phase I were presented and 
discussed. As background for this study, the two papers presented in 1995 will 
be reviewed, since they form the foundation for this paper. 

An important component of the study was the methodology used to create the 
data base. Specifically, all public land-grant and research universities were 
invited to participate in the study. Each institution was asked to send unit record 
files of their entering 1988 freshman class. Using IPEDS definitions for first- 
time, full-time freshmen, records were created for each student, which included 
high school GPA, SAT or ACT scores, gender, ethnicity, class rank, residency, 
four and five year graduation and persistence status. The files were built 
according to a format defined by the authors. These files were then sent to the 
authors’ university using either FTP or e-mail as the method of transfer. The 
files from each of the participating institutions were then merged to form a file 
with over 160,000 records from some fifty-three institutions. A program was 
written that edited the files as they came into the server at the authors’ 
institution. At no time in the building of this large data base did the authors have 
to clean or modify any of the data by hand, cutting down significantly data 
cleanup procedures usually necessary before analysis can begin. The 
successful transfer of large data bases and the subsequent building of a very 
large data base using technology found on virtually all campuses has 
implications for data exchanges among institutions and reporting in general. 

The initial research drew on the work of Astin (1993) in which the predictability 
of graduation rates was examined in relation to students’ entry characteristics. 

In the first paper, the authors replicated Astin’s work, specifically for land-grant 
research universities. Student characteristics were regressed on graduation 
rates to produce a predicted graduation rate for each of the institutions in the 
study, which was then compared to the actual rate. In this analysis, the data 
base was composed of some 130,000 student records from 44 universities. 
Graduation and persistence rates (four and five year) were estimated, using 
high school GPA, SAT or ACT scores, gender, and ethnicity. 
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Comparison of the results of Astin’s prediction equation and the equation 
derived from the analysis described above revealed that whereas Astin 
obtained the strongest correlation with four year graduation rates (R=.34), our 
best results were obtained using five year rates (R=.32). The Astin equation 
over predicted four year graduation rates for 93% of the universities in the 
sample. However, prediction of four year graduation and persistence rates 
were essentially equivalent for both equations. These results raise questions 
about the use of Astin’s equation in predicting graduation rates for land-grant, 
research universities. 

A second analysis compared the efficiency of logistic regression equations and 
linear regression (the methodology used in the above analyses). In contrast to 
other reported findings (Dey and Astin, 1993), in each instance, logistic 
regression performed better than linear regression. Residual analysis showed 
a better fit, particularly at the extremes. Sixty-eight percent of the universities 
had a closer fit between their actual and predicted rates using a logistic 
regression model. This analysis indicated that, although it adds to the 
complexity to the analysis and the interpretation, logistic regression may be 
better than linear regression for predicting graduation rates. 

A third analysis was conducted to examine the impact of university level 
variables on the ability to predict graduation rates. Some twenty-two variables 
were identified for inclusion in the study. Because of the number of variables 
and their disparate nature, factor analysis was used to simplify the data. Using 
an orthogonal rotation, six primary factors were identified from the twenty-two 
variables. Adding factor scores from this analysis to the student background 
variables improved the prediction of graduation rates somewhat. The overall 
logistic regression equation correctly predicted 70.2% of the students’ 
graduation status, with the correlation between predicted and actual rates of 
graduation being .35. Fifty-eight percent of the university graduation rates were 
more accurately predicted using both student and institutional variables than 
with only student variables. 



Methodology 

In this part of our study, Phase II, we were interested in gathering data from 
Land Grant, Research I and AAU universities updated with a more recent cohort 
(1990) and including information about the program students chose upon entry 
and the program from which they graduated. CIP codes, which all universities 
use for federal IPEDS reporting, were gathered from each participating 
university. This database enabled us to update our previous findings and to 
extend our research by analyzing program level information. We answered a 
series of questions about graduation rates both at the university and program 
levels, as detailed in the results section below. 
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Two approaches were used. First, descriptive statistics were used to tabulate 
and summarize the data from a variety of perspectives. Understanding the data 
in a simple manner is both important in its own right and essential before 
proceeding to more sophisticated analyses. To simplify the data, ACT scores 
were converted to the SAT scale using ETS concordance tables (SAT scores 
are on the old scale, not recentered). Also, we converted high school class 
ranks to high school grade point averages for those universities that had only 
ranks by using a concordance table we developed from data in this study using 
a method developed by Chisholm (1993). 

Second, based on student entry characteristics, we calculated and compared 
predicted and actual graduation rates using logistic regression. This extension 
of Astin’s work (Astin, 1993; Dey and Astin, 1994) provided performance 
indicators by calculating predicted graduation rates against which actual rates 
were compared. The independent variables were high school grade point 
average, SAT score, sex, ethnicity and domicile; the dependent variable was 
whether the student had graduated after five years. This analysis was done 
both at the program and university levels. We plan to use the results of this 
analysis to identify universities with much higher than predicted rates in 
particular academic disciplines for future qualitative study. 



Results 



Descriptive Statistics 

What were the summary statistics for the study ? 

Individual student data was collected on more than 204,000 freshmen entering 
38 public, land grant, Research I universities in 1988 and 1990. Table 1 
provides a list of the participating universities. As shown in Table 2 (sorted by 
graduation rate), the five year graduation rate for all of the students was 54.8% 
with an additional 9.9% still enrolled but not yet graduated. The mean SAT 
score was 1029; mean high school grade point average was 3.28; 49.4% were 
female; 14.9% were minority; and in-state residents made up 76.9% of the 
freshmen. 



Did graduation rates vary bv academic program and bv university ? 

Yes, much more by university than by program. Table 3 displays program level 
data showing that five year graduation rates vary from 58.3% for freshmen who 
entered social sciences and interdisciplinary programs to 48.2% for those who 
entered health related professions. Table 2 shows a much larger variation 
among universities, from 25.7% for university #8 to 77.1% for #38. To illustrate ' 



the variation of graduation rates within a specific set of programs, Table 4 
details data in the sciences and math, showing a variation from a low of 29.2% 
for #8 to 85.5% for #38. It is also noteworthy that within sciences and math, 

women had a 61.2% five year graduation rate, while the rate for males was onlv 
54.1%. y 

Did time to completion vary bv academic program ? 

Yes. Table 3 shows that engineering students take longest--16.3% are still 
enrolled after five years--whereas business students finish fastest-only 7.3% 
remain enrolled after five years. A consequence of this is the underestimation 
of graduation rates for engineering students when looking out five or even six 
years. 

What were the student migration pa tterns across programs ? 

Of the 112,000 graduates in the database, 26.0% graduated in business fields 
and 22.5% in the social sciences (see Table 5). These were also the programs 
that experienced the largest in-migration from other areas (each gained about 
15% from other programs or undecided students). Engineering had the least 
amount of “swirling”, while liberal arts and social sciences had the most. As we 
defined it: swirling = [(number of out-migrants) + (number of in-migrants)] / 
(number who graduated from the same program as they entered). 

Since the study gathered six digit CIP codes for 93,447 graduates, we were 
also able to assess the number of students who changed majors between entry 
and graduation. Almost three of every four entering freshmen (72%) who 
initially chose a major-undecided students were excluded-changed to a 
different major before graduating. 

1? there an interaction between pro gram and university: do some universities 
dQ-relativelv better at graduating students in some disciplines ? 

No, the university rankings on five year graduation rate were highly correlated 
across programs— a university with a high graduation rate in one program was 
likely to have a high rate in other programs (r >.95 for seven out of the nine 
comparisons). 

Did academic preparation v a ry among programs and universities ? 

Yes, students entering education programs had the lowest average SAT score 
at 949, whereas engineering students were highest at 1103. High school grade 
point averages had the same pattern. To put this into context, the average 
student entering education would have been at the 16th percentile in 
engineering and the 22nd percentile in sciences and math. Table 3 details 
these results. 
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Preparation varied considerably among universities, as well, ranging from an 
average SAT of 933 at university #8 to 1 108 at #38 (see Table 2). The average 
freshman at #8 would have been at the 1 2th percentile at university #38. 

Looking at a specific area, the average entering sciences and math students at 
#8 would have been at only the 7th percentile at university # 38. Our data, then, 
indicate considerable variation in academic preparation across both programs 
and universities. 

How did other student characteristics vary among programs ? 

As shown in Table 3, females were most likely to enter education (76%) and 
least likely to enter engineering fields (18%). Minority students were most likely 
to enter sciences and math (22.9%); least likely to enter education (7.2%). 
Underrepresented minorities, which we define as African American, Hispanic 
and Native American students, were most likely to enter health related 
professions (13.1%) and least likely to enter education (7.2%). Asian 
Americans were most likely to enter sciences and math. These findings are 
consistent with other national data. 

Were undecided students at risk ? 

No, in fact undecided students, who were about 26% of the study population, 
perform slightly better than students who choose a major upon entry--they 
graduated at a slightly higher rate (56.9% compared with 54.8%), and their 
graduation did not appear to be delayed (a five year graduation and 
persistence rate of 66.5% compared with 64.7% for “decided” students). Also, 
undecided students had very similar entry characteristics, including high school 
preparation, to other students. 

This finding is significant given the mythology about undecided students. 

Clearly undecided students are not poorly prepared upon entry, do not drop out 
at higher rates, and do not take longer to graduate. 



Predicting Graduation Rates 

As discussed earlier in this paper, in our previous study of graduation rates 
(Kroc, Woodard, Howard and Hull; 1995), Astin’s (1993) model was used to 
account for the influence of student background characteristics on graduation 
rates. In this study we extended the logistic regression model used in Phase I of 
our research by including an additional, more recent, cohort (1990) with the 
original 1988 freshman cohort, and by adding program level data (six digit CIP 
codes) that could be used to predict students’ graduation rates within individual 
programs or program areas. 



JdO W well could university graduation rates be nreriinteri nsinn entry 
characteristics ? 

Using logistic regression, we obtained about 66% concordance-students’ were 
correctly classified as having graduated or not 66% of the time, where 50% 
would be chance level. This translates into a correlation of about .28 between 
actual and predicted graduation rates. 

What does this mean? As an example, look at university #1 in Table 2. If you 
knew nothing about this university, your best guess about its graduation rate 
would be the mean of the sample, 54.8%, which was much lower than this 
university’s actual rate of 68.2%. This is the kind of logic naive readers might 
use when looking at graduation rate data from the upcoming Student Right-to- 
Know reports or, currently, when perusing US News and World Report or the 
grad rate data in the NCAA Report. Using entry characteristics, however, the 
predicted rate for university #1 was 64.0%, closer to its actual rate-and a better 
representation of what its actual graduation rate should be. 

The best predictor was high school grade point average, followed by SAT, sex, 
ethnicity and domicile, respectively. In Table 6, to illustrate some of these’ 
relationships in tabular form, we have aggregated the universities into either a 
high, medium or low group based on their graduation rates. Differences in SAT 
and high school grade point averages among these groups are most evident. 

Clearly, the results shown in Table 2 show that entry variables can help us 
understand some of the large variations in graduation rates among universities. 
Examining the differences between actual and predicted rates, then, can help 
us sharpen our thinking and refine our questions about why university 
graduation rates vary. 

Were there differences in the p r edictability of graduation rates at the program 
level ? 

No, we found little variation in either the strength of the relationship or the 
coefficients for the independent variables among the regressions produced for 
each program. The equations were essentially interchangeable. Given the 
degree of “swirling” among undergraduates, this is probably not a surprising 
finding. Table 7 displays the beta weights and the concordance values for each 
program. 

.Did individual univers ities excel in some programs, but not in others ? 

Again, generally, no. The correlation coefficients were about .95 when 
comparing the differences between actual and predicted graduation rates for 
universities across programs (see Table 8). A university with better than 
predicted graduation rates in one program was likely to have better rates for its 
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other programs. The exception in these data was health related professions, 
but much smaller numbers of students in this area may have been the primary 
reason for the lower correlation. 

Are there differences in actual graduation rates among universities with similar 
predicted graduation rates ? 

One of the purposes of this study was to identify, at the program level, 
universities that graduated more students than might be expected from their 
entry characteristics. These universities would then be studied more thoroughly 
using qualitative, case study, methods in Phase III of our project. In particular, 
we would be looking for the influences of university “environmental" 
characteristics on graduation rates. 

Table 9, sorted by predicted graduation rate, shows results for sciences and 
math programs. Examining pairs of universities with similar predicted rates, 
some interesting differences are evident. For example, universities #16 and 
#17 both had high predicted rates but differed by almost 16 percentage points 
in their actual graduation rates for science and math students. University #34 
and #38 showed a similar pattern. University #15 and #27 were also 
interesting--#27 had a lower predicted rate, but higher actual rate than did #15. 



Implications 

These results suggest that there may be value in further studies of the particular 
conditions at individual universities that may influence graduation rates. 
Accounting for entry characteristics as best we could with this methodology, 
there appeared to be variance in graduation rates that was unexplained. 
Moreover, we found intriguing differences within specific programs among 
universities that appeared to have similar entering students. 

Some of the differences and unexplained variance in graduation rates could be 
attributed to limitations in the methodology, specification error in particular. The 
variables used to account for “input” differences among universities do not 
perfectly measure what they purport to measure, and do not represent all of the 
dimensions of student difference. Socio-economic status, for example, is 
probably not adequately measured in this study. Nonetheless, we believe that 
this methodology does move us considerably closer to understanding 
differences in graduation rates that are caused by university culture and 
environment. 

How do we answer the question posed in the title of this paper: “Graduation 
rates: Do students’ program choices make a difference?" Our answer is a 
definitive yes--and no. We found that student characteristics varied 
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considerably among programs, with nearly a standard deviation of difference 
between the low and high programs. But, program-level graduation rates for 
individual universities were almost entirely predictable from the university’s 
overall graduation rate-universities with the highest total graduation rates also 
tended to have the highest rates in their individual programs. And when we 
accounted for entry characteristics using logistic regression, universities with 
better than predicted overall graduation rates also tended to have better rates 
for their individual programs. 

Comparing one university with another, then, the variations among programs 
that occur at the graduate level are not evident at the undergraduate level. This 
is probably not surprising given the impact of lower division course work, when 
students are often taking a common core of courses; the fact that nearly three 
out of four students change majors; and the large number of students who are 
undecided when they matriculate. Overall admission criteria and the overall 
university environment appear to be more important than the selection of a 
program among undergraduates. 

Finally, we believe that the findings of this study bear directly on this year’s AIR 
Forum program theme, performance indicators. For public, Research I, Land 
Grant and AAU universities, the data in this study can provide valuable 
benchmark and comparative information. As we face increasing pressures to 
improve-and report on-undergraduate education, this information may be 
useful. 
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Table 1 



National Graduation Rate Study 
Participating Universities 



Arizona State U. 
Auburn U. 
Clemson U. 

Iowa State U. 
Louisiana State U. 
Mississippi State U. 

Montana State 
N. Carolina State U. 
North Dakota State U. 
Oklahoma State U. 
Penn State U. 
Rutgers U. 

South Dakota State U. 
SUNY at Buffalo 
Texas A&M U. 

U. of Arkansas 
U. of Tennessee 
U. C. Irvine 
U. C. Santa Barbara 
U. of Arizona 
U. of Colorado-Boulder 
U. of Connecticut 
U. of Georgia 
U. of Idaho 
U. of Iowa 
U. of Kansas 
U. of Maine 
U. of Massachusetts 
U. of Minnesota 
U. of Missouri 
U. of Montana 
U. of New Hampshire 
U. of New Mexico 
U. of Oregon 
U. of Vermont 
U. of Washington 
U. of Wisconsin-Madison 
Virginia Tech U. 



Table 2 

National Graduation Rate Study - Combined Data From 1988 and 1990 Freshmen Cohorts 
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